Frugal Optimization for Cost-related Hyperparameters

The increasing demand for democratizing machine learning algorithms calls hyperparameter optimization (HPO) solutions at low cost. Many have hyperparameters which can cause a large variation in the training But this effect is largely ignored existing HPO methods, are incapable to properly control cost during process. To address problem, we develop new cost-frugal solution. core of our solution simple but randomized direct-search method, provide theoretical guarantees on convergence rate and total incurred achieve convergence. We strong empirical results comparison with state-of-the-art methods AutoML benchmarks.